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What is transparent malware analysis? 


> Analyzing the malware without being aware. 


> “Transparent” means that the malware cannot 
detect it. 
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Why transparency is important? 
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What is the current state of malware 
analysis systems? 
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Application App 


Operating System 
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> Unarmed to anti-virtualization or anti-emulation techniques. 


K Large performance overhead. 
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Application App App 
Operating System Malware Analyzer 


Hypervisor/Emulator 


> Unable to handle malware with high privilege (e.g. rootkits). 
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What makes a transparent malware 
analysis system? 
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K An Analyzer which is responsible for the further analysis of 
the states. 
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K An Environment that provides the access to the states of the 
target malware. 


> It is isolated from the target malware. 


> It exists on an off-the-shelf (OTS) bare-metal platform. 


> An Analyzer which is responsible for the further analysis of 
the states. 


> It should not leave any detectable footprints to the outside of 
the environment. 
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System Management Mode (SMM) [1] is special CPU mode 
existing in x86 architecture, and it can be used as a hardware 
isolated execution environment. 


> Originally designed for implementing system functions (e.g., 
power management) 


> Isolated System Management RAM (SMRAM) that is 
inaccessible from OS 


> Only way to enter SMM is to trigger a System Management 
Interrupt (SMI) 


> Executing RSM instruction to resume OS (Protected Mode) 
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ARM TrustZone technology [2] divides the execution environment 
into a secure domain and a non-secure domain. 


> The RAM is partitioned to secure and non-secure regions. 

> The interrupts are assigned into the secure or non-secure 
group. 

> Secure-sensitive registers can only be accessed in secure 
domain. 


> Hardware peripherals can be configured as secure access only. 
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K The Performance Monitor Unit (PMU) [3, 4] leverages a set 
of performance counter registers to count the occurrence of 
different CPU events. 


K The Embedded Trace Macrocell (ETM) [5] traces the 
instructions and data of the system, and output the trace 
stream into pre-allocated buffers on the chip. 


> The PMU exists in both x86 and ARM architecture while the 
ETM is ARM special hardware. 
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Debugging Client Debugging Server 


1) Trigger SMI SMI 
g handler j 
inspect 
2) Debug command | Breakpoint application 
GDB-like Debugged 
Debugger application 
3) Response message 


26 


S 


MalT = Performa nce WAYNE STATE 


UNIVERSITY 


> Testbed Specification 
> Motherboard: ASUS M2V-MX_SE 


>» CPU: 2.2GHz AMD LE-1250 
> Chipset: AMD k8 Northbridge + VIA VT 8237r Southbridge 


> BIOS: Coreboot + SeaBlOS 
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Table: SMM Switching and Resume (Time: jus) 


Operations Mean STD 95% Cl 


SMM switching 3.29 0.08 (3.27, 3.32] 
Command and BP checking 2.19 0.09 [2.15, 2.22] 
Next SMI configuration 1.66 0.06 [1.64, 1.69] 
SMM resume 4.58 0.10 [4.55, 4.61] 


Total 11.72 
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K High performance overhead on mode switch. 
> Unprotected modified registers. 


> Vulnerable to external timing attack. 
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> Use TrustZone as the isolated execution environment. 


> The debug subsystem is similar to MalT while the trace 
subsystem is immune to timing attacks. 


> Modified registers are protected via hardware traps. 
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> Testbed Specification 
>» ARM Juno v1 development board 


> A dual-core 800 MHZ Cortex-A57 cluster and a quad-core 700 
MHZ Cortex-A53 cluster 


> ARM Trusted Firmware (ATF) [6] v1.1 and Android 5.1.1 
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Table: Performance Scores Evaluated by CF-Bench [7] 


Native Scores 


Java Scores 


Overall Scores 


Mean Slowdown Mean Slowdown Mean Slowdown 
Tracing Disabled 25380 18758 21407 
Instruction Tracing 25364 1x 18673 1x 21349 1x 
System call Tracing 25360 1x 18664 1x 21342 1x 
Instruction Tracing 6452 4x 122 154x 2654 8x 
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Table: Time consumption of domain switching (Time: jus) 


ATF Enabled Ninja Enabled Mean STD 95% CI 
x x 0.007 0.000 [0.007, 0.007] 
v x 0.202 0.013 [0.197, 0.207] 


/ J 0.342 0.021 [0.334, 0.349] 
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> OS-related tracing requires software-based approach to fill 
semantic gaps, which involves performance overhead. 


> Malware may intentionally enable the ETM or PMU to detect 
the analysis system. 


> Hardware traps can only protect the system instruction access 
to the registers. 
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> We present MalT and Ninja, malware analysis systems in x86 
and ARM architectures aiming for higher transparency. 


> We consider the hardware-based approach provides better 
transparency than software-based approaches. 


> To build a fully transparent malware analysis system, we are 
seeking for more hardware support. 
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